Goto

Collaborating Authors

 replicate ai study


Researchers are struggling to replicate AI studies

#artificialintelligence

The problem: Science reports that from a sample of 400 papers at top AI conferences in recent years, only 6 percent of presenters shared code. Just a third shared data, and a little over half shared summaries of their algorithms known as pseudocode. Why it matters: Without access to that information, it's hard to reproduce a study's findings. That makes it all but impossible to benchmark newly developed tools against existing ones, causing difficulties in knowing which direction in which to push future research. How to solve it: Sometimes a lack of sharing may be understandable--say, if intellectual property is owned by a private firm.


Researchers are struggling to replicate AI studies

#artificialintelligence

The world's most powerful rocket may be good for more commercial missions than Mars supply trips. One astronomer says it could open access to lots of asteroids on which humans could strike it rich mining metals. Backstory: Earlier this month, SpaceX successfully launched its Falcon Heavy rocket. It's twice as powerful, and costs a quarter the price to launch, as its closest competitor, Delta IV Heavy. What's new: Falcon Heavy's power could get humans to more asteroids to tap them for supplies.


Scientists can't replicate AI studies. That's bad news.

#artificialintelligence

The field of artificial intelligence (AI) may soon have to face a ghost that's haunted many a scientific field lately: the specter of replication. For a research study to be considered scientifically robust, the scientific method says that it must be possible for other researchers to reproduce its results under the same conditions. Yet because most AI researchers don't publish the source code they use to create their algorithms, it's been largely impossible for researchers to do that. Science magazine reports that at a meeting of the Association for the Advancement of Artificial Intelligence (AAAI), computer scientist Odd Erik Gundersen shared a report that found only six percent of 400 algorithms presented at two AI conferences in the past few years included the algorithm's code. Only one in three shared the data they used to test their program, and just half shared a summary that described the algorithm with limited detail -- AKA "pseudocode." Gundersen says that a change is going to be necessary as the field grows.